I. Field of the Invention
This invention relates generally to digital computing and switching systems, and more particularly to a method and apparatus for distributing clock pulses to multiple clocked digital devices with little or no skew resulting.
II. Discussion of the Prior Art
A stable oscillator is an essential element of any digital system employing sequential logic. Digital computers, both general purpose and special purpose, are examples of sequential logic systems. Special purpose digital devices such as switches, multiplexors and device controllers may also employ sequential logic. A typical sequential logic digital system contains a single oscillator, known as the master clock, which controls the sequential operation of all of the digital logic which comprises a given system. The clock distribution system is the means by which the master clock signal is transformed, replicated and distributed to each clocking site in the system.
The leading edge of each clock cycle typically captures the results of the logical operations performed in the previous cycle such that these results become the source of input data for the logical operations to be performed in the present cycle. Proper operation requires that logical results propagate to every clocking site before the corresponding clock signal arrives. From the foregoing it can be understood that the ultimate throughput, i.e., the number of logical steps per second, of any sequential logic system is constrained by both the propagation delay of the combinatorial logic circuits and the uncertainty regarding the time of arrival of the clocking signal at the many clocking site distributed throughout the system. The difference between earliest and latest possible time of arrival of all clock signals is defined as clock skew. It is a common practice to minimize clock skew to thus allow the largest possible fraction of each clock cycle to be available for useful computational work.
A clock distribution system is typically comprised of a master oscillator, wave shaping circuits which establish specified on and off periods for the clock signal, fan-out circuitry which regenerates and replicates the clock signal, and transmission circuitry which distributes the fanned-out signals to remote clocking sites. Most systems require a multi-phase clock which is a set of clock signals having identical wave shape and a specified phase relationship. This discussion will deal only with the distribution of a single phase clock, recognizing that additional phases or subordinate clock signals may be derived after distribution. Also, all comments made regarding the distribution of a single-phase clock are equally applicable to the distribution of a multi-phase clock employing a separate distribution system for each phase.
A clock distribution system is analogous to a wheel comprised of a hub, spokes and rim. The master oscillator and wave shaping circuits are located at the hub. Each spoke represents a divergent transmission path to the remote clocking sites located at the rim. Clock skew arises from the difference in propagation time from spoke to spoke and in an ideal system each spoke would have identical propagation time and negligible degradation of the clock wave shape.
The principal contribution to clock skew is the difference in the electrical length of the transmission circuitry. The electrical length is a function of both the physical length of the electrical conductor and the dielectric medium surrounding the conductor. Electrical signals propagate approximately one foot in one nanosecond in free space, however the propagation time increases as the dielectric constant of the surrounding medium increases. Further an impedance mismatch between the driver circuit and the transmission circuitry may additionally increase propagation time due to signal reflections. In a typical system, the transmission circuitry will include printed circuit foil on the cards which contain the source and destination of the active circuitry, the conductors which comprise the interconnecting backplane and intervening connectors. Since the destination circuitry can be located anywhere in the system, it is inevitable that there will be a disparity in the physical length of the many transmission paths of a typical system. There will be an even greater disparity in the electrical length of transmission paths due to the dielectric differences between the path segments associated with printed circuit boards, connectors and backplane, even when the backplane is itself a multi-layer printed circuit board. The electrical length is further affected by the proximity of the transmission conductors to ground planes and the loading effect of stubs along the transmission path.
Various approaches have been devised for propagating a system clock to digital logic devices populating multiple printed circuit boards. A typical minicomputer shown in FIG. 1, comprises a number of parallel boards, including a CPU card 105, an I/O card 108, and two memory cards 106 and 107. The cards 105 through 108 are interconnected by a backplane 104, with each card connecting to the backplane through a connector 110 having a plurality of terminal pins. The backplane carries data and control signals between the different boards. To make sure that all boards work at the same beat, a special signal called "System Clock" is used to time all of the circuitry on all of the boards. In the example of FIG. 1, the System Clock generator 120, located on the processor board 105, sends the system heartbeat to the circuits 121 of board 105 through the conductive printed circuit traces 113 contained on the board 105, then to the circuits 122 of board 106 through connector 109 of board 105, trace 114 of backplane 104, connector 110 of board 106 and trace 115 of board 106. The generator also sends the heart beat to circuits 123 of board 107 through connector 109 of board 105, trace 114 of the backplane 104, connector 111 of board 107, trace 116 of board 107 and to circuits 124 of board 108 through connector 109 of board 105, trace 114 of backplane 104, connector 112 of board 108, and trace 117 of board 108.
There are two main problems with the prior art clock distribution in the minicomputer configuration illustrated in FIG. 1, namely, clock skew and clock waveform distortion. Concerning clock skew, all clock signals applied to the clock inputs of ICs 121 through 124 must be synchronous or, at least synchronous within an acceptable range. The difference in timing between the clock signals on ICs 121 and 124 is called "clock skew", and a typical value of this clock skew across a system such as is illustrated in FIG. 1 is about 10 percent of the system clock. This corresponds to about four nanoseconds for a system clock running at 25 MHz, corresponding to a 40 ns cycle time. Clock skew, of course, depends upon the difference in clock signal propagation time for the different clock signal paths. This propagation time itself depends on the electrical length of the path and the loading of the clock generator. The path electrical length depends on its physical length and on the different electrical characteristics of the path. In the arrangement shown in FIG. 1, there is a very short physical length of the path from the clock generator 120 to the clock input of IC 121, especially when contrasted to the path length from the clock generator 120 to the IC 124 on the I/O 108. Here, the clock signal must traverse the foil on board 105 leading to the connector 109, through the connector 109 and through the bus 114 on the backplane 104, through the connector 112 for I/O card 108 and, thence, through the printed wiring on that card to the clock input of IC device 124. Assuming a backplane, which is 16 inches in length and that the IC traces on the cards 105 and 108 are each five inches, the path from the clock generator 120 to the device 124 represents an additional physical length of about 26 inches or 2 ns for a typical epoxy backplane. In addition cross the connectors 109 and 112 such that the minimum skew between the two paths is 3 ns.
A significant improvement in reducing clock skew can be achieved using the prior art arrangement shown in FIG. 2 disclosed in an article by Mears entitled "To Clear System Bottlenecks Drive Backplanes with ECL", Electronic Design Bow 35, No. 24, Oct. 15, 1987, pp. 83-88. It shows how the overall distance between the parallel boards can be reduced by placing a clock board 120 on the opposite side of the backplane, now called the centerplane, in that both sides thereof are utilized. The traveling distance of the clock signal via trace 114 is minimized as represented in FIG. 2, but this scheme requires a complex assembly procedure of the connectors that are now soldered on both sides of the backplane.
Another prior art attempt to minimize the skew problem in distributing clock signals from a clock pulse generator to ICs on a series of logic boards is set out in an article entitled "High Chip Density Boosts Performance of Interactive Parallel Super Computer" by John Bond, Computer Design Vol. 29, No. 17, Sep. 1, 1989, pp. 48-50. The scheme disclosed in that article uses an active backplane as is represented in FIG. 3. In this arrangement, active circuitry, such as the clock generator 120, is soldered onto the backplane rather than being disposed on a separate board plugged into the backplane. This offers the advantages of eliminating the need for additional clock distribution board and optimization of the distance between boards. Distances are even shorter if the clock circuitry 120 is mounted on the second side of the backplane, whose first side is occupied by the connectors of the parallel boards.
The drawback to the arrangement shown in FIG. 3 is the attendant complexity in the design of the backplane which now has components on it. This makes it more expensive to manufacture because there must be more traces in the backplane to accommodate the added components. The arrangement shown in FIG. 3 has the additional problem in that the backplane is used as a controlled medium for transmission of clock signals. In practice, the backplane is heavily used for routing a large number of data and control signals and it is very difficult to guarantee that the clock signal will be given precedence over these data and control signals to minimize the miss-match. For example, in a typical minicomputer, such as the one represented in FIG. 1 the backplane 104 would commonly connect ten cards together through connectors having, for example, 400 connects each. This means there are 4,000 connector points or pins. Each pin has to be connected at least to one other pin in the backplane by means of a trace. Thus, in the example under discussion, about 4,000 traces would be required. Because the dimensions of the traces are defined by the backplane technology and cannot be reduced at will, the number of layers of the backplane needs to be increased if it is to support active components thereon. Some backplanes now have up to 60 layers, and their cost is extremely great. If the best possible matching paths for the clock signals through the backplane are to be achieved, a dedicated layer sandwiched between a ground plane and a power plane, must be used if impedance considerations are to be addressed. Increasing the complexity of the backplane by these three layers compounds the cost problem.
Another way of solving the clock distribution problem to minimize skew is to provide an independent clock propagation path outside of the backplane. This can be achieved using discrete cables or discrete optical fibers of equal electrical length running from the clock generating board to the parallel logic boards. The use of cables, however, is not always possible when system size constraints are considered. If the digital system must meet stringent volume requirements, the added bulk attributable to multiple cables used for distributing clock signals from the clock generator to the individual circuit cards can make this an unacceptable solution.